Classification of an item (e.g., a person or subject) to one or another group (e.g., healthy or disease) is a common problem. When measurements of several variables (“predictors”) are available as measurements per item, classical discriminant methods can be used to address this problem. These methods were initiated by Fisher in 1936 and have been extended in different variants since then. Typically, these methods involve: (1) the analysis of a training set of items from which classification functions are derived; (2) these functions are then applied to predictor measurements of a new item, yielding a classification score for each group; and (3) the new item is classified to the group with the highest score. Conventional classification techniques include: Linear discriminant classification analysis, which uses predictor measurements; quadratic discriminant classification analysis (QDA), which uses predictor measurements and their squares and products; Bayesian classifiers, which use adaptable, prior information; and support vector machines, which use non-linear parcellation of predictor space.
There are several limitations of these conventional classification methods. First, these methods may be useful for a relatively small number of predictors (e.g., in the tens), but the methods can become intractable for large predictor sets. A large number of predictors may often be available, but all of the available predictors are not typically used by the conventional methods. In addition, as the number of predictors increases, and if, somehow, all of them can be used, the result is commonly “overfitting”, which means an excellent classification rate of the groups in the training set but a poor classification of new subjects (i.e., poor “external cross-validation”). Another problem is that from a large predictor set, only a small subset might be most useful, and other predictors may simply add noise. So, even if, technically, all of the predictors could be used, this does not necessarily mean that they would improve the classification outcome.